Skip to content

Align inference resolution with javac by applying hints from Maurizio#5004

Closed
stephan-herrmann wants to merge 7 commits into
eclipse-jdt:masterfrom
stephan-herrmann:ivar_ranking
Closed

Align inference resolution with javac by applying hints from Maurizio#5004
stephan-herrmann wants to merge 7 commits into
eclipse-jdt:masterfrom
stephan-herrmann:ivar_ranking

Conversation

@stephan-herrmann

@stephan-herrmann stephan-herrmann commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

Working towards replacing a speculative part of our fix from #4735 with "official" hints from https://mail.openjdk.org/archives/list/compiler-dev@openjdk.org/message/GN6RTCGMME6I5JVLSFZRIR32XY6QKOI2/

Our own tweak from ad3653e has been removed.

No new tests included because we only switch implementation strategies from our own invention to something closer to javac. For known test cases both strategies are indeed equivalent.

@stephan-herrmann stephan-herrmann added the javac ecj not compatible with javac label Apr 16, 2026
@stephan-herrmann stephan-herrmann marked this pull request as ready for review April 17, 2026 22:12
@stephan-herrmann stephan-herrmann added this to the 4.40 M2 milestone Apr 17, 2026
@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

Once again builds fail in a way that's fully opaque to me:

[2026-04-18T10:37:45.561Z] [INFO] --- clean:3.5.0:clean (default-clean) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.561Z] [INFO] 
[2026-04-18T10:37:45.561Z] [INFO] --- tycho-packaging:5.0.3-SNAPSHOT:build-qualifier (default-build-qualifier) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.561Z] [INFO] The project's OSGi version is 1.0.200.v20250210-0737
[2026-04-18T10:37:45.561Z] [INFO] 
[2026-04-18T10:37:45.561Z] [INFO] --- tycho-packaging:5.0.3-SNAPSHOT:validate-id (default-validate-id) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.561Z] [INFO] 
[2026-04-18T10:37:45.561Z] [INFO] --- tycho-packaging:5.0.3-SNAPSHOT:validate-version (default-validate-version) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.561Z] [INFO] 
[2026-04-18T10:37:45.561Z] [INFO] --- target-platform-configuration:5.0.3-SNAPSHOT:target-platform (default-target-platform) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.561Z] [INFO] 
[2026-04-18T10:37:45.561Z] [INFO] --- tycho-compiler:5.0.3-SNAPSHOT:validate-classpath (default-validate-classpath) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.561Z] [INFO] Resolving class path of org.eclipse.jdt.core.tests.builder.mockcompiler
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- resources:3.5.0:resources (default-resources) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/eclipse.jdt.core-Github_PR-5004/org.eclipse.jdt.core.tests.builder.mockcompiler/src/main/resources
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- tycho-compiler:5.0.3-SNAPSHOT:compile (default-compile) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] Compiling 1 source file to /home/jenkins/agent/workspace/eclipse.jdt.core-Github_PR-5004/org.eclipse.jdt.core.tests.builder.mockcompiler/target/classes using Eclipse Compiler for Java(TM) 3.46.0.v20260418-1002
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- tycho-bnd:5.0.3-SNAPSHOT:process (default-process) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- tycho-ds:5.0.3-SNAPSHOT:declarative-services (default-declarative-services) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- tycho-apitools:5.0.3-SNAPSHOT:generate (generate) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- resources:3.5.0:testResources (default-testResources) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/eclipse.jdt.core-Github_PR-5004/org.eclipse.jdt.core.tests.builder.mockcompiler/src/test/resources
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- tycho-compiler:5.0.3-SNAPSHOT:testCompile (default-testCompile) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- tycho-packaging:5.0.3-SNAPSHOT:update-consumer-pom (default-update-consumer-pom) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- tycho-source:5.0.3-SNAPSHOT:plugin-source (plugin-source) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] Building jar: /home/jenkins/agent/workspace/eclipse.jdt.core-Github_PR-5004/org.eclipse.jdt.core.tests.builder.mockcompiler/target/org.eclipse.jdt.core.tests.builder.mockcompiler-1.0.200-SNAPSHOT-sources.jar
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- tycho-packaging:5.0.3-SNAPSHOT:package-plugin (default-package-plugin) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] Building jar: /home/jenkins/agent/workspace/eclipse.jdt.core-Github_PR-5004/org.eclipse.jdt.core.tests.builder.mockcompiler/target/org.eclipse.jdt.core.tests.builder.mockcompiler-1.0.200-SNAPSHOT.jar
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- tycho-p2-plugin:5.0.3-SNAPSHOT:p2-metadata-default (default-p2-metadata-default) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:45.818Z] [INFO] No baseline version org.eclipse.jdt:org.eclipse.jdt.core.tests.builder.mockcompiler:eclipse-plugin:1.0.200-SNAPSHOT
[2026-04-18T10:37:45.818Z] [INFO] 
[2026-04-18T10:37:45.818Z] [INFO] --- javadoc:3.12.0:jar (attach-javadocs) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:48.357Z] [INFO] Building jar: /home/jenkins/agent/workspace/eclipse.jdt.core-Github_PR-5004/org.eclipse.jdt.core.tests.builder.mockcompiler/target/org.eclipse.jdt.core.tests.builder.mockcompiler-1.0.200-SNAPSHOT-javadoc.jar
[2026-04-18T10:37:48.615Z] [INFO] 
[2026-04-18T10:37:48.615Z] [INFO] --- tycho-p2-extras:5.0.3-SNAPSHOT:compare-version-with-baselines (compare-attached-artifacts-with-release) @ org.eclipse.jdt.core.tests.builder.mockcompiler ---
[2026-04-18T10:37:48.872Z] [ERROR] Baseline and reactor have the same fully qualified version, but different content
[2026-04-18T10:37:48.872Z] different
[2026-04-18T10:37:48.872Z]    META-INF/ECLIPSE_.RSA: present in baseline only
[2026-04-18T10:37:48.872Z]    META-INF/ECLIPSE_.SF: present in baseline only

I didn't touch that test plugin, nor do I see any activity in that plugin other than the bump of parent versions in 1c8cd97.

  • no baseline version was found
  • the baseline has different content from the reactor????

I'm not inclined to investigate this build failure.

FYI @iloveeclipse @jarthana

@srikanth-sankaran

Copy link
Copy Markdown
Contributor

May be you are impacted by #5013 (comment) too ?

@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

May be you are impacted by #5013 (comment) too ?

Thanks for the hint. So "my" build failure seems to be caused by eclipse-platform/eclipse.platform.releng.aggregator#3772 and would then need eclipse-platform/eclipse.platform.releng.aggregator#3784

@iloveeclipse

Copy link
Copy Markdown
Member

@stephan-herrmann : please simply rebase your PR on master.

@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

@stephan-herrmann : please simply rebase your PR on master.

Done. I'd be surprised if <failIfNoTests> fixes "Baseline and reactor have the same fully qualified version, but different content", but let's see

@HannesWell

Copy link
Copy Markdown
Contributor

I'd be surprised if <failIfNoTests> fixes "Baseline and reactor have the same fully qualified version, but different content", but let's see

Glad to surprise you :)
But in fact, that is just a fix for a follow-up issue resulting from
#5012, which aims to fix the first issue.
Sorry for the trouble, but now everything should work again.

@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

OK, after build trouble got resolved, back to inference business:

Despite the confirmation from #4937 (comment) I'm not quite ready to merge this, because test success depends on one tweak (from 22dc3e1) which I'm uneasy about.

The tweak introduces use of SAME-bounds during the 2nd attempt of inference resolution and some 30 tests currently depend on it (look for code comment // NON-JLS: leverage existing same bounds if they become proper by substitution:).

I came back to this tweak when addressing #4937: in 3cc73e6 I added code to BoundSet.addBound(TypeBound, LookupEnvironment) to make ivar dependencies bidirectional, but surprisingly including SAME bounds in the game conflicted with the previous tweak regarding SAME-bounds.

@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

Most of the tests needing the tweak from 22dc3e1 are test for record type inference.

For a simple example I debugged RecordPatternTest.testIssue1336_1

				record R<T> ( T t) {}
				public final class X {
					private static boolean foo(R<?> r) {
						if (r instanceof R(String s)) {
							return true;
						}
						return false;
					}
				}

We start from innocent ivar dependencies (ivars corresponding to α1 and β1 in 18.5.5):

	Dependency T#0 = (capture#1-of ?)#0
	Dependency (capture#1-of ?)#0 = T#0

Then resolution creates fresh type variables to yield (partly into incorporation):

	Dependency (capture#1-of ?)#0 = T#0
	TypeBound  (capture#1-of ?)#0 = Z#0-T#0
	TypeBound  (capture#1-of ?)#0 = Z#1-(capture#1-of ?)#0
	Dependency T#0 = (capture#1-of ?)#0
	TypeBound  T#0 = Z#0-T#0
	TypeBound  T#0 = Z#1-(capture#1-of ?)#0

At this point we generate a new, failing constraint:

⟨<Z#0-T#0> = <Z#1-(capture#1-of ?)#0>⟩

We cannot equate two independent fresh type variables. Actually, resolving (capture#1-of ?)#0 isn't even necessary, but I see nothing that would allow us to skip it. Also waiting for one ivar to be instantiated before resolving the second doesn't seem to be an option.

Since record pattern inference raises a few questions of its own, I tried to isolate matters, by letting only record pattern inference use the tweak in question. At this point only two tests fail due to disabling the tweak:

  • org.eclipse.jdt.core.tests.compiler.regression.GenericsRegressionTest.test434118()
  • org.eclipse.jdt.core.tests.compiler.regression.GenericsRegressionTest_9.testIssue4864()

Both tests use enums in special ways. Is that an indication for the cause??

@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

When the dust in this area settles I'll also revisit my NON-JLS invention in #5059

@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

At this point only two tests fail due to disabling the tweak:

  • org.eclipse.jdt.core.tests.compiler.regression.GenericsRegressionTest.test434118()

Here are some fresh results from comparison with javac using latest from #4838:

I used the test reduced to the method where ecj (with pending changes) and javac differ:

public class X {
    public Object convertFails(Class<?> clazz, String value) {
        return Enum.valueOf(clazz.asSubclass(Enum.class), value);
    }
}

ecj and javac agree up until before final incorparation:

javac:

Bound set after Invocation Applicability Inference (B2): 
T <: java.lang.Enum<T>
T = capture#378 of ? extends U
capture#378 of ? extends U <: U
U = java.lang.Enum

ecj:

Reduced all to:
Inference Context (initial) (strict)
Inference Variables:
	T#1	:	NOT INSTANTIATED
	U#0	:	java.lang.Enum
	(? extends U#0)#2	:	NOT INSTANTIATED
Type Bounds:
	Dependency T#1 = (? extends U#0)#2
	Dependency T#1 <: java.lang.Enum<T#1>
	Dependency U#0 :> (? extends U#0)#2
	TypeBound  U#0 = java.lang.Enum
	Dependency (? extends U#0)#2 = T#1
	Dependency (? extends U#0)#2 <: U#0
Local Capture Bounds: <empty>
Other Capture Bounds:
	Class<(? extends U#0)#2> = capt(Class<? extends U#0>)

Notational differences:

  • javac output does not discriminate inference variables from the type variables (@srikanth-sankaran would that information be available internally?) - otherwise I will just have to make sure that all type variables in test programs are uniquely named, despite the tradition to name all type variables <T> ;-P
  • javac shows ivars for wildcards as captures, at least these are numbered so we can translate this to ecj's notation like (? extends U#0)#2 (second ivar, which represents the shown wildcard).
  • ecj shows more bounds in inverse direction

Following this status behavior significantly differs! javac bluntly states:

Bound set after incorporation: <Ditto>

By contrast, ecj performs quite some incorporation work. In particular we have

  • capture bound Class<(? extends U#0)#2> = capt(Class<? extends U#0>)
  • combine this with (? extends U#0)#2 <: java.lang.Enum<T#1> previously created by regular incorporation
  • together this creates U#0 <: java.lang.Enum<T#1>
  • incorporate with the above TypeBound U#0 = java.lang.Enum
  • results in ⟨java.lang.Enum <: java.lang.Enum<T#1>)
  • this reduces to FALSE (unless the constraint would be "soft" which is not the case here).

At this point our bound set is

Type Bounds:
	Dependency T#1 = (? extends U#0)#2
	Dependency T#1 <: java.lang.Enum<T#1>
	Dependency T#1 <: U#0
	TypeBound  T#1 <: java.lang.Enum
	Dependency U#0 :> (? extends U#0)#2
	Dependency U#0 :> T#1
	TypeBound  U#0 = java.lang.Enum
	Dependency U#0 <: java.lang.Enum<T#1>
	TypeBound  U#0 <: java.lang.Enum
	Dependency (? extends U#0)#2 = T#1
	Dependency (? extends U#0)#2 <: U#0
	Dependency (? extends U#0)#2 <: java.lang.Enum<T#1>
	TypeBound  (? extends U#0)#2 <: java.lang.Enum
Local Capture Bounds: <empty>
Other Capture Bounds: <empty>

This no longer resembles what javac tersely declares as "ditto".

@srikanth-sankaran when debugging javac on the example in this comment, can you say more about this sequence:

  • "Bound set after Invocation Applicability Inference (B2):"
  • "Bound set after incorporation: <Ditto>"

Is there really no further activity from incorporation?

@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

Anyway, the above comparison hinted at where javac and ecj start to go separate paths, and even if information from javac after that point may be incomplete, I found a plausible tweak that actually fixes the two intermediate regressions: mark constraints derived from capture bounds as "soft".

@srikanth-sankaran

Copy link
Copy Markdown
Contributor

This no longer resembles what javac tersely declares as "ditto".

@srikanth-sankaran when debugging javac on the example in this comment, can you say more about this sequence:

  • "Bound set after Invocation Applicability Inference (B2):"
  • "Bound set after incorporation: "

Is there really no further activity from incorporation?

Yes, the serialized textual representation of the boundsets at those points are identical.

Improve the impl to cover one more issue
+ really make ivar-ivar dependencies bidirectional
+ also consider ivar-super bounds in ivar prioritization
  - but not so for same bounds

Also fix an NPE in debug logging

Fixes eclipse-jdt#4937
+ only during regular inference:
  + record inverse of same-bounds for use by rankIVar()
+ only during record pattern inference:
  + workaround for inverse same-bounds in rankIVar()

Still need to always use same-bounds during 2nd attempt of resolution

Minor corrections (moving code lines up/down)

Failures in
+ GenericsRegressionTest.test434118()
+ GenericsRegressionTest_9.testIssue4864()
  - GenericsRegressionTest.test434118
  - GenericsRegressionTest_9.testIssue4864
+ more propagation of isSoft flag (for consistency)
+ slightly more debug output
+ make debug configurable per system property
@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

org.eclipse.jdt.core.tests.compiler.regression.GenericsRegressionTest.test434118()

Even with today's installment from #4838 javac shows much less incorporation activity than ecj.

In particular ecj combines these

        Dependency T#1 = (? extends U#0)#2
        Dependency U#0 :> (? extends U#0)#2

to produce

Dependency T#1 <: U#0

This is where we start to connect bounds using Enum with bounds using Enum<T#1> which probably caused the failure before making more bounds "soft".

For comparison, javac has (among others):

             T = capture#702 of ? extends U
             capture#702 of ? extends U <: U

in my book this directly implies

T <: U

Not so in javac.

Latest tracing gives a few incorporation steps:

  • SubstBounds[undet=U,t=java.lang.Enum,bound=EQ]
    • produces capture#702 of ? extends U <: java.lang.Enum
  • SubstBounds[undet=capture#702 of ? extends U,t=capture#560 of ? extends U,bound=EQ]
    • instantiates capture#702 of ? extends U kind = CAPTURED instantiation = capture#560 of ? extends U
    • we don't see where capture#560 comes from. Is javac performing regular capture of a non-proper type?
  • SubstBounds[undet=T,t=capture#560 of ? extends U,bound=EQ]
    • instantiates: Inference variable = T instantiation = capture#560 of ? extends U

In particular this doesn't look like the pairwise combination of type bounds required by JLS.

@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

Let me guess: capture#702 of ? extends U is not fully treated as an inference variable?

  • true, it is special as it appears in a capture bound
  • but that doesn't mean it should not participate in incorporation, does it?

Aha, inside Infer.PropagateBounds I see plenty of if (undet.hasTag(TypeTag.UNDETVAR) && !((UndetVar)undet).isCaptured()). So javac excludes CAPTURED inference variables from some incorporation work.

More specifically, a capture bound in ecj may look like this:

        Class<(? extends U#0)#2> = capt(Class<? extends U#0>)

Here (? extends U#0)#2 is one of the fresh beta ivars mentioned in 18.5.2.1. right where capture bounds are defined. From how I read JLS , those betas should be just plain normal ivars.

Javac trace shows:

TraceOut: Inference variable = capture#702 of ? extends U kind = CAPTURED 
TraceOut:     capture#702 of ? extends U <: U

To me this looks like

  • capture#702... is treated as the ivar (beta) introduced in 18.5.2.1
  • it also represents the said capture bound, so capture bounds are defined at the level of type arguments rather then the parameterized type mentioned in JLS.
  • yet by being marked CAPTURED incorporation doesn't treat it as an ivar? That looks wrong.

@stephan-herrmann

Copy link
Copy Markdown
Contributor Author

I created a consolidated version of this PR as #5129 and out-sourced some left-overs to #5130

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

javac ecj not compatible with javac

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants